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ABSTRACT 

Social e-learning is a process through which learners achieve their learning goals via social interactions with each other 
by sharing knowledge, skills, abilities and educational materials. Adaptive e-learning enables adaptation and 
personalization of the learning process, based on learner needs, knowledge, preferences and other characteristics. In this 
paper, we present a case study that analyzes the social interaction features of a social personalized adaptive e-learning 
system developed at the University of Warwick, called Topolor. We discuss the results of a quantitative case study that 
evaluates the perceived usefulness and usability. The results demonstrate a generally high level of learner satisfaction 
with their learning experience. We extend the discussion of the results to explore future research directions and suggest 
further improvements for the studied social personalized adaptive e-learning system. 
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1. INTRODUCTION 

The Internet and the World Wide Web have made it possible to provide new types of learning environments 
where learners can interact with their peers and engage in effective and attractive learning experiences 
(Welsh, 2003). Social media is comprised of Internet-based applications that stand on the ideological and 
technological foundations of Web 2.0. These applications allow creation and exchange of user generated 
contents like never before (Kaplan, 2010). The increasing use of social media and Web 2.0 tools as well as 
various social features such as sharing, tagging, rating, commenting in e-learning systems can offer new 
opportunities for communication, collaboration, and active participation in a learning process (McLoughlin, 
2011). Discussions and group work are often integrated into collaborative and participative learning practice, 
providing a range of educational benefits, which are thoroughly discussed in the literature (e.g. (Hrastinski, 
2009), (Rovai, 2004) and (Woo, 2007)). 

Adaptive (Educational) Hypermedia (A(E)H) (Brusilovsky, 2004) is another research direction that offers 
improvements to the area of e-learning. AEH systems (e.g. (Cristea, 2007), (Ghali, 2009) and (Foss, 2009)) 
provide personalized learning experiences to individual learners, according to a range of characteristics, such 
as learning goals, background knowledge and preferences (Rosmalen, 2009). The use of adaptation, along 
with the social affordances of Web 2.0 tools, carries a great potential for improving e-learning systems and 
learning experiences. However, the review of the previous work indicates that current e-learning systems 
have only marginally explored the integration of social interaction features and adaptation techniques. This 
research intends to address this gap by evaluating a system that was developed to foster effective social and 
adaptive e-learning experiences. 

The aim of this research is to improve learning experience and learning outcomes via a social adaptive 
learning paradigm, based on the hypothesis that extensive social features, personalized recommendations and 
Facebook-like appearance of a system, anticipated to make the environment more familiar to learners, 
subsequently increases the usefulness and usability of the system. To isolate research variables, this paper 
focuses exclusively on studying the usefulness and usability of the social features in adaptive social 
e-learning. It is based on our recent quantitative case study that explores the use of Topolor (Shi, et al., 
2013a; Shi, et al., 2013b) - a social personalized adaptive e-learning system. The rest of paper will review 
the research background, depict the social interaction tools in Topolor, and present the conducted experiment 
and its results. And finally, the conclusions and the outline of future work will be described. 
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2. RESEARCH BACKGROUND 

Learning is inherently a social experience. Social aspects of learning have been emphasized in a range of 
theoretical frameworks developed to explain how people learn (e.g. (Vygotsky, 1978) and (Wenger, 2009)). 
Yet, developing effective and efficient online social learning environment remains an open problem. While 
online interaction via social networking services has become widely accepted and heavily embedded in 
day-to-day life, providing solutions that foster creation of effective e-learning spaces are not straightforward. 

Online interaction tools have also been integrated and used in AEH (Brusilovsky, 2004). This enabled 
adaptation of educational hypertext to the personalized needs of learners (Brusilovsky, 2000) in e-learning 
systems. Since early 2000s, many AH frameworks have been proposed, such as AHAM (Wu, 2002), the 
Munich model (Koch, 2006), XAHM (Cannataro, 2002), LAOS (Cristea, 2003) and GAF (Knutov, 2008). 
Few were later extended to accommodate some social features, e.g.. Social LAOS (SLAOS) (Ghali, 2009b) 
added a collaboration mechanism into the framework, and led to the development of the MOT 2.0 system 
(Ghali, 2009c). It introduced social features such as a chat tool, tagging, rating and commenting on learning 
content. However, while these systems cater for personal needs within specific learning contexts, they are 
often limited in their strategies for adapting to social needs or in their social features. Some recent work 
(Simko, 2010) has already proposed the need for creating adaptive and highly interactive integrated learning 
environments. However, their work suggests only a limited number of mechanisms for enabling social 
interaction. Hence, there is a gap for extending and evaluating social interaction tools in adaptive e-learning 
settings. Additionally, their framework does not take into account the role of learner familiarity with other 
social interaction tools from e-learning environments and social networking websites, such as Facebook. 

In fact, the features that many of the learners are familiar with from social networking websites remain 
missing from the current e-learning systems. For instance, sharing a learning status, engaging in a simple 
question/answer exchange and sharing notes remain cumbersome or impossible in many of the available 
systems. Subsequently, the potential of adaptation, recommendation and personalization that is based on the 
use of the above social features remains largely unexplored. 

In this paper we address the above gap by introducing and evaluating a range of social features previously 
missing from the available adaptive e-learning systems. 


3. THE TOPOLOR SYSTEM 

To evaluate our hypothesis (SECTION 1), based on our experiment of requirement analysis (Shi, et al., 2012), 
we have developed the Topolor system built on the Yii Framework 1 and Bootstrap 2 . Topolor is made available 
open source and is hosted on Github 3 for easy sharing and version control. It is deployed 4 and used as a social 
learning environment to support some postgraduate level modules in the University of Warwick. The 
registration for using the Topolor system has been recently opened to public. Thus, a larger cohort of users is 
expected in the near future, providing opportunities for collecting feedback, usage data and suggestions for 
further improvements. 

The Topolor system mainly consists of three sub-systems (Figure 1). Each of these contains a set of 
interaction features that are generally referred to here as the social interaction toolset (Shi, et al., 2013c). The 
subsystems of Topolor are the following: 

• Topolor-Home provides a chronological list of the learning statuses posted by individual learners. It also 
provides access to a set of interaction tools that encourage informal communication and collaboration 
such as commenting on, sharing and favoring of learning statuses (Figure la). 

• Module Center offers a warehouse of online courses, as well as provides adaptive learning content 
recommendation, learning expert recommendation, and interaction tools that encourage personalized 
social e-learning such as sending messages to recommended learning experts (Figure lb and Figure Id). 


1 http://yiiframework.com 

2 http://twitter.github.com/bootstrap 

3 http s :// github .com/ aslanshek/topolor 

4 http://www.topolor.com 
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• Q&A Center maintains some lists of questions/answers related to the learning contents, and provides 
adaptive question recommendation, learning topic recommendation, expert peer adaptation and social 
interaction tools for discussions and practices (Figure lc). 



c. Q&A Center d. Module Center - Quit for a Topic 

Figure 1. User Interface of Topolor 


The social interaction toolset is one of the most important components in the Topolor system. To provide 
easy access to interaction, this toolset can be accessed from many places in the system. For instance. Figure 2 
shows the interaction with this toolset from the Topolor-Home index page (Figure la). 

This paper focuses on three social interaction tools. The status tool (Figure 2a) is used to share learning 
statuses. Learners can favorite and comment on each other’s posted learning statuses; the messaging tool 
(Figure 2b) is used to send private messages to others; and the Q&A tool (Figure 2c) is used to ask and answer 
questions. Learners can also use Q&A tool for discussions. 


Menage QUA Note Todo 


Status Message ES Note Todo 


a. create a Status 

Status 122221 Q&A Note Todo 

u«ti >"••»•«« » Wt S3 

|^J Cm C3 “■ 

b. send a Message c. ask a Question 


Figure 2. Social interaction toolset: (a) learning status creation tool; (b) messaging tool; (c) Q&A tool 


4. CASE STUDY DESIGN 

Topolor has been evaluated from various perspectives (Shi, et al . , 2013d). In this section, we present the 
design of the conducted case study. The case study was comprised of three consecutive stages: 1) the 
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experiment of using the Topolor system, 2) the questionnaire about the usefulness and ease of use, and 3) the 
analysis of the questionnaire results and some qualitative feedback from the users. 

4.1 Experiment Design 

The experiment was conducted with the help of 21 students from the Department of Computer Science at the 
University of Warwick, who were registered for a 4th year MSc level module ‘Dynamic Web-Based 
Systems’, and a lecturer who was leading this module. The experiment lasted for 2 hours, during which the 
students were asked to learn a lesson on ‘Collaborative Filtering’ from the system as well as ensuring to 
perform specific tasks to familiarize themselves with the features related to the provided social interaction 
toolset. The full list of the 18 tasks completed by the students is listed in Table 1. 


Table 1 . Tasks Performed by The Student 


Status 

Message 

Q&A: Question 

Q&A: Answer 

Create (1) 

Send (7) 

Create (9) 

Create (12) 

Edit (2) 

Reply (8) 

Edit (10) 

Edit (13) 

Remove (3) 


Remove (11) 

Remove (14) 

Comment on (4) 


Share (15) 


Favorite (5) 


Favorite (16) 


Share (6) 


Add Tag (17) 
Edit Tag (18) 



4.2 Questionnaire Design 

Usefulness and ease of use are fundamental determinants of user acceptance for a tool usage (Adams, 1992). 
After running the experiment, each student was asked to fill in a questionnaire to measure usefulness and ease 
of use of the social interaction toolset. Likert Scale (Mclver, 1981) questions were used to get the feedback on 
all available features, as tested by tasks in Table 1. The students had to select one of the five responses for 
usefulness and ease of use, respectively, as shown below. A score was assigned to each response, on a scale of 
1-5 as numbered below. After collecting the questionnaires, the responses of the students were analyzed. 

• usefulness : 1) very useless, 2) useless, 3) neither useless nor useful, 4) useful, and 5) very useful. 

• ease of use: 1) very hard, 2) hard, 3) neither hard nor easy, 4) easy; and 5) very easy. 


5. ANALYSIS AND RESULTS 


Out of the 21 students who participated in the experiment, 10 students responded to the optional questionnaire. 
The results extracted from the questionnaire are presented below. Figure 3 a shows the mean of the responses, 
and Figure 3b shows the standard deviation. The number of total questions in the questionnaire was 36, of 
which 18 questions were for testing usefulness and 18 for ease of use. 


5 



(1) (2) (3) (4) (5) (6) (7) (8) (9)(10)(11X12X13)(14X15)(16)(17X18) 


TASK 

a. mean of the responses for each task 


1.2 

1 

0.8 

0.6 

0.4 

0.2 

0 



(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)(11)(12)(13)(14)(15)(16)(17)(18) 
TASK 


b. standard deviation of the responses for each task 


Figure 3. The a. mean and b. standard deviation of the responses for each task (Y-axis presents rating scores; X-axis 
presents the task order; blue columns present the usefulness; red columns present the ease of use) 
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5.1 Usefulness 

The blue columns in Figure 3a and Figure 3b present the usefulness results. The means of the summative 
results rank between 3.7 and 4.6. The standard deviations of the overall results are between 0.516 and 0.994. 
All the reported values of a mean are much larger than 3 (the neutral response), suggesting students’ attitudes 
to be generally positive. 

5.2 Ease of Use 

The red columns in Figure 3a and Figure 3b present the ease of use results. The means of the overall results 
rank between 3.8 and 4.7. The standard deviations of the overall results are between 0.483 and 1.135. As all 
the means are greater than 3, it enables us to infer that most of the students found the social interaction toolset 
to be relatively easy to use. 

5.3 Reliability 

We adopted Cronbach’s alpha to measure the reliability of the test. According to Carmines, a Cronbach’s 
Alpha of 0.8 is considered as highly reliable (Carmines, 1979). The values of Cronbach ’s Alpha for each of the 
questions are shown in Table 2. Both usefulness and ease of use are considerably larger than 0.8, suggesting a 
high level of reliability of the results. 


Table 2. Cronbach’s Alpha (Reliability Statistics) 



Cronbach’s Alpha 

Cronbach’s Alpha Based on 
Standardized Items 

Number of Items 

Usefulness 
Ease of Use 

.934 

.948 

.944 

.957 

18 

18 


6. DISCUSSION 

In addition to the questionnaire data collected from the students, we also received some qualitative feedback 
from both students and the lecturer of the module. The general feedback was consistent with the results of the 
questionnaire. However, the responses included some specific suggestions for further improving some of the 
social interaction features, which ranked lower from the point of usability and ease of use. Due to the space 
limitation, this paper focuses mainly on the quantitative results from the questionnaire. However, some of the 
qualitative feedback is discussed below as appropriate. 

Overall, the results from the questionnaire demonstrate that the social interaction toolset is perceived to be 
useful and easy to use. 83% of all the features have been rated by the students as useful, and 78% of the 
features as easy to use (i.e., average mean > 4). Consistently with the questionnaire, qualitative feedback 
included a description of the system as “similar to known social networking sites (e.g. Facebook); fast and 
responsive”. Another respondent said: “One of the best aspects of Topolor is the ability to interact with others 
during the process of learning”. We now proceed to a detailed discussion of the individual social interaction 
tools, namely status, messaging and Q&A. 

6.1 Status 

The questionnaire results demonstrate that the feature (4), commenting on a status, was rated as the third most 
useful feature (mean = 4.5), and its ease of use was ranked as the fourth highest (mean = 4.5) among all the 
social interaction features. This result is further supported by the qualitative feedbacks. For example, one of 
the respondents explicitly mentioned that commenting on each other’s statuses was one of his favorite features 
for interacting with other students. 

On the other hand, (5) favoring a status had the lowest rating (mean = 3.7) on usefulness. The possible 
reason for this could be that the students might not have known what the use of favoring a status was. We 
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assume that it would be necessary to develop a mechanism for providing basic information on less familiar 
features such as favoring. Additionally, wider use of favoring with other features such as questions/answers 
might also affect the future patterns of use. Furthermore, the possible reason for the second lowest rating on 
(5) favoring a status for its ease of use (mean = 3.9), can be that labels for favoring/unfavoring statuses 
became visible only when the status message was being hovered over. The suggested improvement would be 
to keep the labels and the number of times the statuses are favored always visible. 

6.2 Messaging 

The rating for (7), sending a message, was, whilst high, the second lowest (mean = 3.9) with regards to its ease 
of use. The possible reason for this is the current notification mechanism for new messages. More specifically, 
if a student was on the messaging page, on receiving a new message a notification button, like ‘You have 2 
new messages’, would show up. Clicking on that button would refresh the webpage with an AJAX response 
and show the received messages on the top of the message list. However, if the students were on other 
webpages, they wouldn’t know whether they received any new messages. Therefore, the students might have 
had no idea when and how to start messaging. Additionally, whilst most of the webpages in the Topolor 
system provide at least one tool for sending messages to other students, such as 1) a new message box (Figure 
2b) and 2) an avatar list of the recommended learning peers (Figure 4a) that could be clicked on, and then a 
messaging box (Figure 4b) would pop-up. There are still other webpages that did not provide such tools, 
potentially affecting the results on the ease of use of sending messages. 

1 Answer* - 

Alexandra Send message to: lonny 

15 answers(s) 

jonny 

7 answers(s) 

dana 

3 answers(s) 

usern |j^ Cancel 

2 answers(s) 

a. avatar list b. messaging box 

Figure 4. a. Avatar list of recommended learning peers to send messages to; b. the resulting pop-up messaging box 

6.3 Asking and Answering Questions 

The questionnaire results indicated that (12) answering a question was rated as the most useful feature (mean 
= 4.6) as well as the easiest feature to use (mean = 4.7), among all the social interaction features. A similar 
result was found from the qualitative feedback, where the way of asking and answering questions was 
explicitly mentioned as favorable. Furthermore, (9) asking a question was rated very high on the usefulness 
(mean = 4.5) and ease of use (mean = 4.5) too. Therefore, we can report with confidence that the students were 
very satisfied with features of asking and answering questions. 

However, the usefulness of (18) editing the tags of a question was rated as the second lowest (mean=3.8), 
and the usefulness of (17) adding tags to a question was rated as the fourth lowest (mean=4. 1). It seems that 
tagging on questions was not considered as useful as other features of the Topolor system. We can conjecture 
that when a student asked a question in a given learning content area, the relation between the question and the 
learning content would have been automatically established, so that tagging the question would not have 
brought additional benefits. Posting questions beyond the learning content area would be necessary to further 
comment on this feature. 

The ease of use of the feature for asking/answering questions, (10) editing a question was rated the lowest 
(mean = 3.8). To provide an attractive user experience, we used AJAX calls to implement the feature. For 
example, when a student clicked on the title or the description of a question, it would activate the HTML 
editing box; and when the editing box lost focus, it would be replaced by the updated HTML text. No explicit 
buttons were provided to trigger editing actions. This might have not attracted student attention to the 
existence of this functionality. Even though the style of the mouse cursor changes when hovering the title or 
the description of a question, this hint might not have been a clear enough indication to the students about the 
provided editing functionality. Moreover, editing a question may require engagement with the system over a 
longer period of time, so the evaluation of this feature is to be finalized after using the system in the long term. 


la 
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7. CONCLUSION 

In this paper, we have 1) presented the social interaction toolset of the Topolor system, 2) reported a case 
study with a quantitative analysis on its usefulness and ease of use, and 3) discussed the results and identified 
potential improvements of the toolset. The developed Topolor system, as shown in Table 1, was designed to 
include a wider range of social interaction features than previous AEH systems. The results of the case study 
show that the social interaction toolset is found to be useful and easy to use. The overall attitude of the students 
towards the social interaction toolset in particular was very positive. The oral feedback was that they would 
have wanted to have more lessons in it. Decisive in this, we believe, was the fact that a lot of the social 
features had a look and feel familiar to them, similar to the popular Facebook environment, familiarity that is 
essential to consider in designing such systems. 

Although all of the questions received positive responses from the students, we are still keen to improve 
the social interaction toolset further. We reviewed the relatively lower rated features and discussed the possible 
reasons that might have led to lower ratings. To improve this toolset, we intend to conduct further research 
based on the presented results and the discussion, particularly in the following directions: 

• Redesign the favoring tool. First, we intend to find a better way of favoring and un-favoring learning 
statuses, questions, and learning topics. Second, we intend to explore the use of data on favored items for 
adaptation and personalization. 

• Provide a status-filtering tool. We intend to extend the existing mechanism of learning status sharing by 
introducing a support for filtering and evaluating its benefits for collaboration. This feature has the 
potential to improve the process of locating relevant statuses and communicating with learning peers. 

• Provide an auto-tagging tool. While tagging enables to connect various concepts within the system, 
students are usually reluctant to tag. Hence, we intend to enable Topolor to automatically generate tags for 
questions and explore inter-relations. 

• Improve AJAX notification messaging: We intend to identify a more appropriate mechanism for delivering 
system notifications. 

• Enhance the messaging tool. We intend to enhance the messaging tool by supporting instant messaging 
and appropriate notification. 

The development of the next version of the Topolor system has been initiated. Future experiments will 
focus on evaluation of the new and improved features. The future deployment of the enhanced system will 
especially enable further inquiry into the role of social interaction in adaptive e-learning environments and 
the benefits for enriching learning experiences and improving learning outcomes. 
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